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Abstract 



The optimum quality that can be asymptotically achieved in the estimation 
of a probability p using inverse binomial sampling is considered in this paper. 
A general definition of quality is used, in terms of the risk associated with a 
loss function that satisfies certain assumptions. It is shown that the limit su- 
perior of the risk for p asymptotically small has a minimum over all (possibly 
, randomized) estimators. This minimum is achieved by certain non-randomized 

estimators. The model includes commonly used quality criteria as particular 
cases. Applications to the non-asymptotic regime for specific loss functions are 
discussed. 

Keywords: Asymptotic properties, Inverse binomial sampling. Sequential es- 
timation. 

00 ! 1 Introduction 

CO I Inverse binomial sampling is a method for estimating the probability of success, p, in a 

sequence of Bernoulli trials. Given r G N, as many outcomes as necessary are observed 
' until r successes are obtained. The resulting number of trials A'' is a sufficient statistic 

i ( Lehmann fc Ca scUa. 1998i p. 101), from which p can be estimated. 



The uniformly minimum variance unbiased estimator, given by ([Haldand . 119451 ) 



r - 1 

has several interesting properties. Its normalized mean squared error E[(p— p)^]/p^ has 
an asymptotic value for r > 3, namely limp^o E[(p — = l/ (r — 2); and E[(p 



p) ]/p is guaranteed to be smaller than this value for any p e (0,1) (jMikulski fc Smith 



19761) . Similarly, the normalized mean absolute error E[|p — is smaller than its 



asympt otic value, giv en by 2(r — 1)'' ^ exp(— r -I- l)/(r — 2)!, for any p S (0, 1) and 
r > 2 (jMendol . l2009l ). In addition, this estimator, as well as the modified version 



p = {r — 1)/N, can guarantee that for p arbitrary and r > 3, the random interval 
\p/ IJ,i, pp,2\ contains the true value p wit h a confidence level greater than a prescribed 
value ( Mendo fc Hernandol . liooel. l2008allbl ) . 



The results mentioned apply to specific estimators, defined as functions of the 
sufficient statistic N. A natural extension is to investigate whether the quality of 
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the estimation can be improved using other estimators. The most general class of 
estimators is that formed by randomized estimators de fined in terms of N . This 
includes non-randomized estimators as a particular case. Mendo fc Hernandol ( 20091 ) 



address this problem, using the confidence associated with a relative interval as a 
quality measure. It is shown that the confidence that can be guaranteed, either for p 
asymptotically small or for p g (0, 1) arbitrary, has a maximum over all estimators. 
Moreover, non-randomized estimators are given that can achieve this maximum. 

A further generalization is to allow for an arbitrary definition of quality. The 
present paper pursues this direction, focusing on the asymptotic regime. Namely, 
quality is defined as the risk associated with an arbitrary loss function. The allowed 
loss functions are restricted only by certain conditions, which are easily satisfied in 
practice (and which, in particular, hold for all the previously mentioned examples 
of quality measures). Using this general definition of quality, the asymptotic perfor- 
mance of arbitrary estimators in inverse binomial sampling is analyzed. As will be 
seen, the quality that can be asymptotically achieved has a maximum over all estima- 
tors. Furthermore, this maximum can be accomplished using certain non-randomized 
estimators, whose form is explicitly given. 

Section [2] contains preliminary definitions and observations required for the main 
results, which are presented in Section |3l Section |4] discusses these results, and con- 
siders applications in the non- asymptotic regime. Proofs of all results are given in 
Section [5] 

2 Preliminaries 

The following notation will be used. Let fc*^*-* denote k{k — 1) • • • (fc — i + 1), for fc e Z, 
i e N; and k^°^ = 1. The probability function of N, f{n) = P[N = n], is 

I{n)^^-^-^^p^{l-pr-^, n>r. (2) 

The upper and lower (not normalized) incomplete gamma functions are respectively 
denoted as 



T{s,u)= t""' exp(-T) dr, (3) 

J u 

7(s,u)=/ r''-iexp(-r)dT = r(s)-r(s,it). (4) 
In addition, the functions 4>{i') and ijj{x, fl) are defined as 

H^)-'^^^^^^^, -^K^ (5) 



(r-1) 



Given a function h, the one-sided limits limj._j.jj- h(x) and lim^_f.i^+ h{x) are respec- 
tively denoted as h{a—) and h{a+). Given two functions hi, h2 : i— >■ K"'"U{0}, hi{x) 
is 0(/i2(x)) as x — > oo (respectively as a; — >■ 0) if and only if there exist a,M e R+ 
such that hi{x) < Mh2{x) for all a; > a (respectively for all x < a). Similarly, hi{x) 
is Q{h2{x)) as x — oo (respectively as x — >■ 0) if and only there exist a,m,M e R+ 
such that mh2{x) < hi{x) < Mh2{x) for all a; > a (respectively for all x < a). 
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The quality of an estimator p is measured by the risk (expected loss) rj ~ E[L(p/p)] 
associated with a non- negative loss function L : K+ i— > K+ U {0}, provided that this 
expectation exists. The function L is defined in terms of p/p, rather than p. This is 
motivated by the fact that, irrespective of the error measure in use, a given average 
error value is most meaningful when compared with p, and therefore commonly used 
quality measures are most often normalized ones. 

The loss function is assumed to satisfy the following. 

Assumption 1. For any xi,X2 G with X2 > xi, L is of bounded variation on 

[X1,X2]. 

Assumption 2. For any xi,X2 G with X2 > Xi, L has a finite number of discon- 
tinuities in [xi,X2]. 

Assumption 3. The loss function has the following asymptotic behaviour: 

1. There exists K (zM. such that L{x) is 0{x^) as x ^ Q. 

2. There exists K' < r such that L[x) is 0{x^ ) as x ^ oo. 

These restrictions are very mild. Note that the loss function L is not required to be 
convex, or continuous; however, being of bounded variation implies that its discontinu- 
ities can only be jumps or removable discontinuities, i.e. L h as left-hand and right-hand 
limits at every point of its domain, and this limits are finite (jCarter fc van Brunt , I2OOOI 



corollary 2.7.3). All quality measures mentioned in Section[T]can be expressed in terms 
of functions oi x = p/p for which Assumptions [TH3] hold; namely L{x) — {x— 1)^ corre- 
sponds to normalized mean square error, L{x) = |a; — 1| to normalized mean absolute 
error, and 

I 1 otherwise 

corresponds to 1 minus the confidence associated with a relative interval [p//i2,p/zi]. 

Since TV is a sufficient statistic, for any estimator defined in terms of the ob- 
served sequence of Bernoulli variables for which E[L{p/p)] exists, there exists a pos- 
sibly randomized estimato r expressed only in terms of N that has the same risk 
( Lehmann fc Casella . 19981 p. 33). Therefore, attention can be restricted to estima- 



tors that depend on the observations through only; however, randomized estimators 
need to be considered in addition to non-randomized ones. 

The set of all functions from {r,r + l,r + 2, . . .} to is denoted as A non- 
randomized estimator p is defined as p = g{N), with g G T. A randomized estimator 
is a positive random variable p whose distribution depends on the value of TV. The 
distribution function of p conditioned on = n will be denoted as n„. The random- 
ized estimator is completely specified by the functions n„, n > r. Denoting by J-n 
the class of all functions from {r, r -I- 1, r + 2, . . .} to the set of distribution functions, 
a randomized estimator is defined by a function G € T-n that to each n assigns n„. 
Clearly, non-randomized estimators form a subset of the class of randomized estima- 
tors. Throughout the paper, when referring to an arbitrary estimator without speci- 
fying its type, the general class of randomized estimators (including non-randomized 
ones) will be meant. 

The risk will be explicitly denoted in the sequel as a function of p, that is, r\(j)). 
For a non-randomized estimator defined by g € J^, the risk r\(j)) is given by 
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Depending on L, g and p, this series may be convergent or not. In particular, bound- 
edness of g is sufficient to ensure that the series converges for all L satisfying Assump- 
tions [TH3] and for all p. In general, for possibly randomized estimators, 

'7(p) = E/('^) / i(y/p)dn„(y), (9) 

where the integral is defined in the Lebesgue-Stieltjes sense. Assumptions [TH3] assure 
that this integral always exists; however, it may be finite or infinite. Besides, even if 
it is finite for a given p and for all n, the series in ([9]) does not necessarily converge. 
According to this, for an arbitrary estimator and p given, r]{p) may be finite or infinite; 
however, there exist estimators that have a finite risk for all p. 

An arbitrary estimator may not have an asymptotic risk, i.e. limp_^o vip) need 
not exist in general. Therefore, the asymptotic behaviour of an estimator should be 
characterized by lim supp_^Q 77 (p) . The significance of this parameter lies in the fact that 
limsupp_^o '7(-P) the smallest value such that any greater number is asymptotically 
an upper bound of ri{p). That is, given any ryo > limsupp_^Q ri{p), there exists S > 
such that rj{p) < r/o for p < S; and no such S can be found for 770 < limsup^^Q ?7(p)0 

According to the preceding discussion, a desirable asymptotic property of an es- 
timator is that it achieves a low value of lim supp_^o In order to characterize 
how low this value can be, the infimum of limsupp_^Q 77(p) over all estimators should 
be determined. A related question is whether there is an estimator that can achieve 
this infimum. As will be seen, the answer to this question is affirmative, that is, the 
infimum is also a minimum. This implies that there exist optimum estimators from 
the point of view of asymptotic behaviour; moreover, they can be found within the 
class of non-randomized estimators, as will also be shown. To obtain these results, the 
following approach will be used. It will be first shown that for a certain subclass of 
non-randomized estimators, limp_i.o rj{p) exists and can be easily computed. Secondly, 
it will be established that limp^Q ri{p) has a minimum value over the referred subclass. 
Thirdly, this minimum will be shown to coincide with the unrestricted minimum of 
limsup„_^Q ?7(p) over the class of arbitrary estimators. 



3 Main results 

For a given loss function L, the set of all functions g (z T such that Imip^Q rj{p) exists 
for p = g{n) is denoted as J-p. The set of functions g J- for which limn^oc ngin) 
exists, is finite and non-z ero is denoted as -F n - Ob serve that the definition of J^p 
generalizes that given bv iMendo fc Hernandol (|2009t ). which assumes a specific loss 



function, namely ([7]). The result in Theorem [T] to follow establishes that J-n ^ ^p, 
and explicitly gives limp_).o vip)- ^or any g £ with lim„_^oc. ngln) = fi, let 



(l)(v)L{n/v)Av. (10) 



Equivalently, f] can be expressed as 

/•oo 

,7= / il;{x,n)L{x)dx (11) 







^For rjo = limsupp_>Q r)(p) the result may hold or not depending on the estimator and loss function. 
For example, it holds for the estimator JTJ and the loss function corresponding to normalized mean 
squared error, as mentioned in Section [T] On the other hand, it obviously does not hold for an 
estimator equal to a constant value. 
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by means of the change of variable v — Q/x (both expressions will be used in the 
proofs of the results to be presented) . By Assumptions [1] and [3l these integrals exist 
as improper Riemann integrals, and have a finite value. It should be observed (and 
will also be explo i ted in the proofs) that they can also be interpreted as Lebesgue 
integrals (|Apostoi 1 1974 theorem 10.33). 

Theorem 1. Consider r G N. For any loss function satisfying Assumptions]^^ and 
for any non-randomized estimator defined by a function g G J-^, the limit limp^o vip) 
exists and equals fj given by pUj) (or ([TT]) ). 

According to this, the asymptotic risk of an estimator defined by any function 
g G Tn depends on this function only through il, i.e. only the asymptotic behaviour 
of g matters. Furthermore, under an additional assumption, it can be shown that the 
asymptotic risk is a function of fi. 

Assumption 2'. L has a finite number of discontinuities in M^. 

It is evident that Assumption [23 implies Assumption [2j While more restrictive. 
Assumption [23 is satisfied by a large class of loss functions, including the mentioned 
examples. 

Proposition 1. Given r £N, a loss function satisfying AssumptionsUllF} and\^ and 
an estimator defined by a function g e J-^, the asymptotic risk fj is a function of 
Q eR+. with 

d, r^^(-,^U(.,),,. (12) 



Denoting by fj\^. the asymptotic risk corresponding to and r given, this derivative 
can be expressed as 

dii ^ n ■ ^^■^> 

Within the restricted class of non-randomized estimators defined by J-"n, it is natu- 
ral to search for values of fl that yield low values of the asymptotic risk fj. Depending 
on the loss function, there may be or not an optimum value of $7 e M+, in the sense 
of minimizing fj. Theorem [5] to follow establishes that, under certain additional hy- 
potheses (represented by Assumption [4]) , fj indeed has a minimum with respect to 

n. 

Assumption 4. The loss function satisfies the following properties: 
1. There exists ^ G such that L is non-increasing on (0,^) and 

m - L{x) 



rr+l 



dx > 0. (14) 



2. There exists ^' G such that L is non- decreasing on (^',cxd) and one of these 
conditions holds: 

(a) Li^'-)<Li^'+). 

(b ) There is t G N such that L is of class C* on an interval containing ^' and 

d'L 
dx^ 



= /orz = 1,2,... ,t-l, (15) 



> 0. (16) 
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The next Proposition gives a sufficient condition that may help in assessing whether 
a given loss function satisfies property [T] in Assumption |4l 

Proposition 2. // there exist A E M. and B, s such that 

L(x) — A 

hm — — ! = B with Bs < 0, s < r, (17) 

inequality (1141) holds for some ^ G . 

Theorem 2. Given r G N and a loss function satisfying Assumptions[ll [23 a?i(i[2J 
consider the class of non-randomized estimators defined by functions g G J^n- Denoting 
Q = lining oang{n), there exists a value of Q which minimizes the asymptotic risk fj 
among all D, G M+. 

This Theorem indicates that in the stated conditions, and restricted to the class 
defined by J-"n, there is an optimum value of fl from the point of view of asymptotic risk. 
This optimum is not necessarily unique. In the sequel, rj* will denote the minimum of 
fj over the class of estimators defined by J^n, and ^* will denote any value of fl which 
attains this minimum, that is, 

/"OO 

?7* = / (t){v)L{n*/iy)diy. (18) 
Jo 

Assumption 0] holds for a wide range of loss functions, and in particular for those 
corresponding to normalized mean square error, normalized mean absolute error, and 
confidence associated with a relative interval. It is not difficult, however, to find a loss 
function for which the Assumption does not hold, and for which fj does not have a 
minimum over the class defined by J^n- For example, given Ai,A2 > 0, let 

{0 if a; G [l/^l2,^J■l], 
A2 ifx<l/fi2, (19) 
Ai if a; > /ii, 

which is a generalized version of ([7]). Substituting ([T9| into ([T4l) . it is seen that 
property [T] in Assumption |4] is satisfied if and only if 

4^ < (A^l/^2)^ (20) 

while property [2] holds irrespective of Ai and A2. On the other hand, for O G M+, 
substituting ([TO)) into ([TU)) and computing dfj/dQ gives 

dfj _ r?''^^ (Ai/ij"''exp(-n/^i) - ^2/^2exp(-n/X2)) 

dn~ (r-1)! • ^ ' 

This implies that fj has a single minimum at 

^ ^ rlog(^i^2) -log(^iM2) 

^J■2 - i/a*i 

This value is positive if and only if (1201) . or equivalently property [T] in Assumption [31 
is satisfied. Thus, if this property does not hold, fj is monotonically increasing for 
fl G M+, which implies that there is not an optimum Q within R+. 
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Under the hypotheses of Theorem [2l the optimum value of ft for the considered r, 
i.e. r2*, satisfies, by Proposition [1] 

<-> 

(or equivalently, using the notation in the referred Proposition, jylfi^.r — f7|n*,r+i)- 
Thus if (|23|) has only one solution, it must be equal to ft*. If there are several solutions, 
at least one corresponds to the absolute minimum of fj, although not necessarily all of 
them do. 

According to Theorem [2l if the loss function satisfies Assumptions [l] [23 [3] and IH 
any non-randomized estimator defined by a function g e J-^ with lim„_i.oo ng{n) = ft* 
minimizes limsupp_j.g ry(p) within the restricted class of estimators defined by J-^', but 
not necessarily within the class of arbitrary non-randomized estimators, or within 
the general class of possibly randomized estimators. However, under slightly stronger 
conditions this turns out to be true, as established by the next Theorem. 

Assumption 3'. The loss function has the following asymptotic behaviour: 

1. There exists K < r such that L{x) is Q{x^) as x Q. 

2. There exists K' < r such that L{x) is Q{x^ ) as x oo. 

Assumption |33 replaces Assumption |31 in the sense that each property in Assump- 
tion 12] implies the corresponding one in Assumption [31 The new conditions are only 
slightly more restrictive, and are still satisfied by a large set of loss functions, in par- 
ticular by those previously mentioned as examples. 

Theorem 3. Given r € N and any loss function satisfying Assumptions[J[ [J] and 
[2[ limsupp^o ^(p) ^'^'5 ^ minimum over the general class of estimators defined by T-n, 
and this minimum equals rj* . 

Corollary 1. Under the hypotheses of Theorem [3 any non-randomized estimator 
defined by a function g € J^n with \im.n^oa ng{n) = ft* minimizes limsupp_jQ 77(p) 
among all estimators based on inverse binomial sampling. 

Theorcm[3]and Corollary [T] show that, under the stated Assumptions, an estimator 
can be found within the class defined by J^n that is asymptotically optimum over the 
general class represented by T-ji. 



4 Discussion and applications 

4.1 Significance of the results. Non- asymptotic regime 

Since p is unknown, it is desirable to have an estimator that guarantees that the risk is 
not larger than a given 770 for p arbitrary, or at least for all p within a certain interval; 
that is, such that 7y(p) < 770 for P within some interval (^1,^2)7 with < pi < p2 < 1- 
If pi — 0, the estimator is said to asymptotically guarantee that the risk is not larger 
than ryo; ifj in addition, pi = 1, it globally guarantees that the risk is not larger than 

Th e results presented in Section[3]generalize the asymptotic analysis in lMendo fc Hernando 
( 2009h . which considers the specific loss function ([7]), to arbitrary functions satisfying 
the indicated Assumptions. The importance of these asymptotic results lies not only 
in the fact that in many applications p is small, but also in the observation that asymp- 
totic behaviour sets a restriction on the risk that can be guaranteed. This restriction 
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is represented by the followin R Proposition, which is a straightforward generahzation 
of Mendo fc Hernando ( 20091 proposition 1), and its Corollary. 



Proposition 3. // an estimator has a risk ri{p) not larger than a given tjq for all 
P G {Pi,P2), then necessarily limsupp^p^ T]{p) < tjq for any po E [pi,P2]- 

Corollary 2. Given r G N and a loss function that satisfies AssumptionsUl E] and 
[2J for any ijq < 77* and p2 > 0, no estimator can guarantee that rjijp) < 770 for p < P2- 

According to the results in Section[3l if Assumptions H] [2l [3l and l4l are satisfied, any 
estimator defined hy g € J-"n with lim„_j.oo ng{n) — f2* can asymptotically guarantee 
that the risk is not larger than rj* + e for any e > 0, whereas Corollary [5] states that 
no estimator exists with this property for e < 0. It remains to be seen if there exist 
estimators that asymptotically guarantee that r]{p) < rj*; and, particularly, if this 
guarantee can be global. The answer to these questions depends on the loss function 
under consideration. Since a general analysis seems impracticable, a separate study 
needs to be carried out for each loss function. Several important cases, including the 
loss functions already mentioned as examples, are discussed next. 



4.2 Confidence 

For the loss function given by ([7]), ri{p) equals 1 — c(p), where c(p) — P[p/ ^2 < P < 
p/ii] = P[p/ 111 < p < PlJ-2] is the confidence associated with a relative interval defined 
by ^1,^2 > 1- Let c* — 1 ~ rj* , which represen ts the maximum con f idenc e that could 
be guaranteed to be exceeded. The analysis bv lMendo fc Hernandol ( 20091 ) shows that 
assuming r > 3, the inequality c{p) > c* can indeed be asymptotically guaranteed for 
any ^1, /Z2, and globally guaranteed if /ii, ^2 satisfy certain conditions. 



4.3 Mean absolute error 



For L{x) — \x~ 1|, risk corresponds to normalized mean absolute error. Considering an 
estimator p = g{N) with lim„_>.oo ng{n) — CI, and for r > 2, (jlOp gives the asymptotic 
risk 



V 



1 



2(r(r,r2) - or(r - 1,0)) n 

(r - 1)! ^ r - 1 



1, 



(24) 



and it is straightforward to show that (|23l) reduces to r(7' — — {r — 2)!/2. This 
equation has only one solution, which thus corresponds to 57*. Interestingly, for p = 
Q,* /{n — 1) with r > 2, numerically evaluating rj{p) suggests that this estimator may 
globally guarantee rjip) < rj*. However, proving this conjecture remains an open 
problem. 



4.4 Mean square error 

The function L{x) — (x — 1)^ corresponds to normalized mean square error. This 
loss function lends itself easily to non- asymptotic analysis. Considering an estimator 
p = g{N) with lim„_>oo ng{n) = f2, and assuming r > 3, (jlOp gives 

and thus (j23p has the single solution O = r — 2, which is the optimum value for O, 
i.e. O*. From ([25]) the resulting 77* is 1/(7'— 1). As established by the next Proposition, 
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an estimator can be found that globally guarantees that the risk is not larger than 77* , 
namely 

P^H^ (26) 

Proposition 4. Given r >3, and for any p G (0, 1), the estimator (|26p satisfies 

< 1 . (27) 

r — 1 

The following Corollary is obtained from Theorem [3] and Proposition |4l 

Corollary 3. For r > 3, the estimator ()26|) minimizes sup^g^Q ^-j E[(p — p)^]/p^ among 
all (possibly randomized) estimators based on inverse binomial sampling. 

Thus the estimator given by ()26|) not only minimizes limsupp^Q E[(p — 
but also suppgjo 1) ^[(P ~ V)^\lv^ 1 it is minimax with respect to normalized mean 
square error. Therefore, from the point of view of guaranteeing that the normalized 
mean square error does not exceed a given value, (|26p is optimum among all estimators 
based on inverse binomial sampling. 

Comparing the estimators ([Ij and ((26)) . the former can only guarantee E[(p — 
vf'XIv^ < ~ 2), whereas the latter guarantees E[(p — _p)^]/p^ < l/(r — 1). This 
better (in fact, optimum) performance is obtained at the expense of some bias; namely, 
it is easily seen that (f26| gives E[p — p]/p = — — 1). 

4.5 A generalization of confidence 

According to Mendo fc Hernando! ( 20091 proposition 3), for the loss function ([7]), and 



assuming that r > 3, /ii > ^/{r — ^/r) and fJ.2 ^ + \fr + 1)/$^, the estimator 

p = — ^ (28) 

globally guarantees that 77(73) is smaller than its asymptotic value 77. Taking into 
account that, in this case, ri{p) = P[p < + P[p > PMi] and that the proof 

given in the cited reference considers P[p < p/fJ.2] and P[p > p^i] separately, it 
can be seen that the same result holds for the loss function with ^1 = or 
A2 = 0. Furthermore, the result can be generalized to any loss function that can be 
approximated as a (possibly infinite) sum of functions of this form. This is the content 
of the next Proposition. 

Proposition 5. Given r > 3 and f2 G M+, consider a loss function for which As- 
sumvtions[2\ W] and^ hold and that satisfies the following: 

1. L is constant on an interval [v,v'], with 

V < j= — -, v' > p. 29 

2. L is non-increasing on (0,^]. 

3. L is non- decreasing on [v',00). 

In these conditions, for any p G (0, 1) the risk rj{p) of the estimator (|28[) satisfies 
Vip) ^ with fj given by (I10|) (or (fTT|) ). 
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It is noted that conditions[TH3]of Proposition[5]iniply that Assumption[T]necessarily 
holds, and also imply that L{v~) > L{v+) and L{y'—) < L{v'+). 

The following result, analogous to Corollary [H is obtained for the estimator 

$1* 



Corollary 4. Given r >3 and a loss function that satisfies AssumvtionsUi \2\ \3] and 
let fl* be as determined by Theorem\^ If conditions {JW^ in Proposition^^ hold for 
some V, v' with 

v< 7= 7, v' > ^, 31 

the estimator (|30p minimizes suppg(-Q j^-j r7(p) among all (possibly randomized) estima- 
tors based on inverse binomial sampling. 

This establishes that, under the stated hypotheses, the estimator is minimax, 
i.e. minimizes the risk that can be globally guaranteed not to be exceeded. 



5 Proofs 

The following definitions will be used: 



^{p^v)^^^-^l-—Y^(v-iv), pe (0,l),i.eM+, (32) 



C= / i,{y)L{VLlv)Av, 17,CTeM+. (33) 

J T ja 

Lemma 1 (jMendo fc Hernandol . l2009l lemma 1). For any v € R+, < (^(y) < I. 

Lemma 2. Given vitV2 G with V2 > vi, for v G [i^i,i^2] function $(p, i/) 
converges uniformly to <i>{v) as p ^ 0. 

Proof. The lemma is equivalent to the result that $(^^,1^) converges uniformly on 
V € \v\^v-2\ for any s equen ce (p^) such that pfe G (0,1), pfc — >■ 0, which is proved in 
iMendo fc Hernandol (|2009l lemma 3). □ 

Proof of Theorem\^ The risk r\{j)) tends to f\ for p — if an d only if 7 i(pfc) converges 
to f\ for every sequence (pfe) such that g (0, 1), pfe — > ( Apostol ll974L theorem 
4.12). Consider an arbitrary sequence of this type. Let = viPk), and let fk denote 
the probability function / for p = Pk. Defining <j>k{i^) = ^{Pk,t^), it is seen from ([2|) 
and ([32]) that fk{n) ^ pk(t>k{npk)- 

From property [T] in Assumption [H there exist if G M and Ml , xl G M+ such that 

L{x) < Mi^x^ for x < XL. (34) 

Without loss of generality, it will be assumed that K < 0. On the other hand, prop- 
erty [5] implies that there exist K' < r and M^, x^ G such that 

L{x) < M^x^' for X > Xl. (35) 
The risk rj'' is expressed from ([8|) as 

.'= = f:/.(n)L(^). (36) 
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Given a, 13 G M+ with f3 > a, the set Ik is defined as 

h = { Hpk\ , [a/Pk\ + 1, . . . , }. (37) 

Under the assumption that 



Pfe < (38) 
r 

which implies that min/^, = [a/p^J > r, the following definition can be made: 

V'o - E M-)L (—) (39) 

The proof will proceed as follows. With a suitable choice of a and (3, and for k 
sufficiently large, the term tjq can be made arbitrarily close to fj, as will be seen. On 
the other hand, the difference t]^ — r]Q will be decomposed as the sum of three terms, 
each of which can be made arbitrarily small for sufficiently large k. Adequate bounds 
will be derived for each of these four terms, and then the bounds will be suitably 
combined to show that r]^ tends to f/ as fc — oo. 

In the following, npk will be denoted as iyn,k- Assuming 

Pk < (40) 

(which obviously implies ([35])), it is easily seen that for n € 1^, Vn,k is contained in 
the interval / given as 



ra ^ a 
,/3 H 



(41) 



Lemma [3] implies that the sequence of functions {4>k ) converges uniformly to 4> for 

V G I; that is, given Cunif > 0, there exists A:unif such that |0fc(i^) — (t){v)\ < eunii for 

V G I, k> fcunif- Thus fk{n) = Pk(j){l'n.k) +PkOnn\f,n with |6lunif,„| < Eunif for fl G /fc , 

k > fcunif- In these conditions, since 4'{i'n.k) > (Lemma[T]), (|39p can be expressed as 

„t?l V 'I^MJ \Pk J 

On the other hand, since ng{n) — >■ as rt — >■ oo, given eost > there exists rtcst > r such 
that \ng{n) - n\ < Cost for aU n > Ucst, i-e. g{n) = {fl + 9cst,n)/n with |6'cst,n| < Ecst- 
Therefore, assuming 

Pk<—: (43) 

"est 

which implies that min Ik > Jiost , (02) can be written as 

V'o = E P^^M (l + |P^) L f^llll^) . (44) 

Denoting = miui^g/ (/'(i^), which is non-zero because of Lemma[T] from (|44l) it stems 
that 

^k = + ^ pk^ML f^l±^) (45) 



for some 6'unif with l^unifl < funif- 
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Assuming Ccst < and taking into account (|40l) . it follows from (|37p that for 

n G /fc, both ^/vn.k and (fJ + 0ost,n)/^'n,fe are contained in the interval /' given as 



3(r + 1)17 



2{(3 + a/{r + l)y 2ra 



(46) 



According to Assumption[51 L has a finite number of discontinuities in J'. Let d denote 
this number. Each of these discontinuities, located at xi , . . . , Xd, may be either a jump 
or a removable discontinuity. Let 



lim L{x) — L{xi) 



lim L{x) — L(xi) 



(47) 



Thus J represents the contribution of all discontinuities to the total variation of L on 
/'. 

The function L on the interval /' can be decomposed as the sum of a continuous 
function and a piecewise constant function the latt e r of w hich has discontinuities 
at By the Heine-Cantor theorem (jApostoll . Il974 theorem 4.47), Lc is 

uniformly continuous on /'. Since |0cst.n| < eost7 it follows that for any econt > there 
exists i5cont such that \Lc{{Vl + 9cst,n)/vn,k) - Lc{^/vn^k)\ < Ccont for Eost < (5cont, for 
all n G Ik, and for all k. Regarding L^, let 



Uk 



n e /fc I Ld ( ^ 

Vn.k I \Vn,k 



For ne Ik\ Uk, 



^ ri + 0cst,n \ ]^( ^ 
Vn,k ) \Vn,k 



^ ^coiit ■ 



(48) 



(49) 



For each n G Uk, \Li{{Q, + 9cst,n)/'^n.k) ^ Ldi^/'^n.k)\ can be at at most J, and thus 

^ + ^cst,n \ , / 



L 



^n,k 



- L 



Vn.k 



< £cont + J- 



(50) 



Let Xk denote the number of elements of Uk divided by that of Ik ■ Taking into account 
that the latter is less than (/3 — a) /pk + 3 < {(3 — a + 3) /pk and that the function (p is 
upper-bounded by 1 (Lemma [1]), from (|49p and ([5(I| it follows that, for ecst < Scant, 



Pk(t>{vn,k)L 



nelk 



n + i 



Vn,k 



^ Pk(j){Vn.k)L 



nelk 



Vn.k 



< {13 -a + 3){econt + J Xk)- 
. 

It is easily seen that limfc_>.oo Xk can be made arbitrarily small by taking ecst sufficiently 
smah. Thus, given edisc, there exist (Jdisc, ^disc such that Xk < Cdisc for ecst < <^disc, 
k > fcdisc- Consequently, for ecst < niin{(5cont, <5disc} and k > fcdisc, 



Pk(t>{Vn,k)L ) - ^ pk(l>{Vn,k)L ( 



< (/?-a-|-3)(econt + Jedisc)- 
(52) 



From (SSI) and (152 



Cunif 



n 



V Pk(t'{Vn,k)L + (^ - a + 3)(6'cont + Jfdisc) 

Ln6/fc 



(53) 
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with |6'cont| < Ccont, |6'disc| < Edisc • The sum over n in tends to (j){v)L{Q./ v) dv 
as fc — )> cxD. Thus for any eint > there exists /cint such that for aU k > hint 



Pk(t>{vn,k)L {—— \ - / <t>{v)L f'— ^ dv 



< Eint) 



(54) 



and therefore (|53p can be expressed for k > max{fcdisc, feint} as 

cont + JOdisc) 
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(55) 



with I flint I < eint- In addition, given any etaii, there exist ataii, Aaii with /3taii > ataii 
such that \fi — (l){v)L{Q./v) dv\ < etaii for < a < ataii, f3 > /?taii- Thus, in these 
conditions, 



Cunif 



[V + fltail + ^'int + {13 - a + 3){0 

cont + "Z^'disc)] 



with l^taill < etaii- 

The difference ry*"' — 1]^ can be expressed as Vi + V2 ~^ V3 j where 



"cat — 1 



n—r 
La/PfcJ-l 



gin) 



Pk 



g(") 
Pk 

Pk 



Regarding the term jyf , from ^ it is seen that 



fk{n) < 



(7^ 



and therefore 



"eat — 1 



T — 1 T ^cst 



n'' ^pI r f 5W A '^cst Pk 
Pk J (r - 1) 



^ V T 
1)1 ^ ^ 



Pk 



(56) 



(57) 
(58) 
(59) 



(60) 



(61) 



The fact that hm„_>.oo ng[n) exists and is finite imphes that the function g is upper- 
bounded by some constant Mg. For g{n)/pk > ^l, ([SS]) implies that L{g{n)/pk) < 
Ml^{Mg/pk)^ ■ On the other hand, g{n)/pk in (|6T|) is greater than rrig = min{(7(r), (7(r-|- 
l), . . . , (7(nost — 1)}; and for g(n)/pfe G {mg,x[2, Assumption [T] imphes that L{g{n)/pk) 
is lower than some value Afg, where both rrig and A/g depend on ricst- Thus, for the 
range of values of n in , 



Pk 



< max • 



,M'A< 



max{M[Mi^ , Ml} 



(62) 
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The sum in the right- most part of (|6l1) is either empty or else it contains riest — r< Uest 
terms. Therefore, using (j62|) . 



^ nl.,^ maxjM; Mf' , M'} 

0<v'[<^ ^r^jf-^pl ■ (63) 

Regarding ri2, the sum in (j58p is empty for a/pk < ficst + 1- If it is non-empty, 
since n > ricst, the term g{n)/pk can be written as {^ + 0cst,n)/i^n.k with |0ost,n| < Ccst- 
Therefore, taking into account (pO]) . 

Since ecst < it holds that 0/2 < O + 6*051, « < 3il/2, and thus for the range of 

values of n in ([58|) 

3r2 ^ -f 6*081,71 ^ ^ ^gg^ 

21^™, fc ^'n,fe 2i/„_fe 2a 

Therefore, assuming il/(2a) > x^, for n within the indicated range it stems from psp 
that 

L ( ^1±^) < Ml ( ^1±^Y < Ml ( ■ (66) 
Substituting ^ into ((M)) . 



n=ncat 

Consider ej^^jj > arbitrary. Since K' < r, defining 

\Ml{3n/2)K' J 

it follows from (|67|) that for any a < ajg^ji 

< ^2 < (69) 

As for 7^3, taking into account that (1 —pkY^''''' < 1/e, from ([2]) and ([5)) it is seen 
that 

< ^51^. (70) 
In addition, (j43p implies that n > nost for any n within the range in (1591) . Thus 

Since Cost £ f^/2, 

f2 + 6*031, n 3f2 3r2 , , 
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Thus, assuming 3f2/(2/3) < xl, and taking into account that K < 0, it stems that for 
n within the indicated range 



< Ml 1^-1^ < Ml _^ . (73) 

VnM J \ Vn,k J \2Vn,kJ 

If it is additionally assumed that pk < 1/2, the factor l/{l—pkY in (|7T|) cannot exceed 
2^ Therefore 

0<^3^< %, E P^<:.''-'-M-n^k). (74) 

The sum in (fM]) tends to r(r — if, /3) as A: — >■ oo. Thus, given e[^^ > 0, there exists 
k'-^^ such that for k > k[^^ 

oo 

0< E PfcCfe'^"'exp(-i/„,,) <r(r-X,/?)+e;„t. (75) 

r/3/pfci+i 

In addition, since r(r — K,l3) is positive and tends to as /3 — ?► oo, for any ej'g^jj > 
there exists such that <r{r- K,l3) < e^'^jj for /3 > /S^'^jj. Therefore d?!]) can be 
written as 

0<^3^< (Z;^), «an + 4t)- (76) 

To establish that 77'^ — fy, it suffices to show that for any eo > 0, there exists fco 
such that 1?]'^ — f]\ < eo for all k > fco- With the foregoing results, and taking into 
account the dependencies between the involved parameters, this is accomplished as 
follows. Given eo > 0, let 

etaii = ^- (77) 
This determines the values ataii and /3taii- Likewise, taking 



tail g 



(78) 



determines a^^^jj, and taking e"g^;j such that 



2-^Ml»^ _eo 

(r-1)! '^'i" 9 ^ ' 

determines The values a and /3 are selected as 

(80) 
(81) 

(Note that, since /3taii > Q^taih dHHl) and ((5T|) imply that (3 > a.) From a and /3, the 
intervals I and /' are obtained, and the values m^, d and J can be computed. Taking 

eint = I (82) 



'taib 


n 1 








*taib 


2^ 



15 



determines fcint- The parameter eunif is selected such that 



- , 4eo\ eunif eo ,oo\ 



which determines fcunif- Next, econt is chosen such that 



(/3-a + 3)eeo„t = ^, (84) 



from which (5cont is obtained. Taking edisc as 



edisc = (85) 



determines Jdisc and /cdisc- Choosing any ecst smaUcr than min{il/2, (5cont, f^disc} de- 
termin 



termines Ucst, from which rUg and can be obtained. Let k^st be such that for aU 



(r-1)! ^'^ "^9- ^ ' 

Let fc^g^ be chosen such that (03]) holds for all A: > k'^^^. , and fcintorv such that holds 
for all fc > fcintcrv The parameter e[^^. is chosen as 



^int *-taib 



(87) 



from (|63l) and (|86| . 



from (El and dlS 



and from (ITS)) . ((75)) and ((57|) . 



which determines fc^'^^^. Finally, let /Cconst be such that Pfc < 1/2 for all fc > fcconst- Tak- 
ing ko = max{ fcint, fc(„t,fcunif,fccst,fccst:fcdisc,fcintorv,fcconst}, the followiug inequalities 
are obtained for fc > fco. From ^ and (|82 l) - ((85)) . 

: k 4eo / 4eo\ eu„if 5eo ,„„^ 



0<r7f<f; (89) 

o<%'<|; (90) 

0<%^<f^. (91) 

Inequalities ([88|) -([9T |) imply that \t]'' — fj\ < eq for all fc > fcg, which concludes the 
proof. □ 

Proof of Proposition [7J By Assumption [23 let D be the number of discontinuities of 
L, occurring at points xi < X2 < ■ ■ ■ < xd- The asymptotic risk fj can be expressed 
as X^i^o "^itb 

fjo= ij{x,n)L{x)dx, (92) 
Jo 

fji^ ilj{x,n)L{x)dx, i^l,...,D-l, (93) 

J Xi 

fjo^ ij{x,n)L{x)dx. (94) 



16 




Given z — 1, . . . , D — 1, let Li(x) be defined for x G [x^, x^+i] as 

Xi <C X x-^-j-i, 
= { L{x^+), X ^ X,, (95) 

X — 

and let Ti{x,V,) be defined for x G O G M+ as Ti(x, fi) = ■i(j{x,il.)Li{x). 

Clearly, the integral in (|93p does not change if ip{x, ft)L{x) is replaced by Ti{x, ft). The 
function is continuous on [x^jXi+i] x M"*", because it is the product of continuous 
functions . The function dTi/dVt is similarly seen to be continuous. This implies 
( Flemin j . 19771 corollary to theorem 5.9) that f]i given by is a function of i7, 
with 

d,._ r-^^^(-,^).d. (96) 



di7 J^^ on J^^ dn 

Regarding f]o, let To{x, fl) = V'(x, n)L{x) for x e (0, 17 e M+, and To(0, 17) = 

0. It is clear that Tq is continuous on (0,xi] x M+. In addition, its continuity at any 
point of the form (0, Hq) can be established as follows. Let A be any value such that 
< A < fio- For D, e {^0- A, + A) and a; > 0, Tq is bounded as 

< ro(x, n) < ■ (97) 

Property[l]in Assumption |3] implies that the right-hand side of ((97)) tends to as x — >■ 0. 
Thus there exists 6 > such that < To{x, Q) < e for < x < 5, \Q - Qo\ < A. 
This shows that Tq is continuous at (0, flo), and thus on [0, xi] x M+. Using analogous 
arguments, dTo/dfl can also be seen to be continuous on [0,a;i] x M+. This implies 
that 770 is a function of fl, and ([M)) holds for z = if the lower integration limit is 
replaced by 0. 

As for fj]j, let T{x,il) = 4'{x,^)L{x), and consider the function T {x , fl) / il"^ . This 
function and its partial derivative with respect to fl are continuous on {xu, 00) x IR+, 
and satisfy the following bounds: 

< ^ ' ' < ' (98 

a;'-+i(r-l)! ^ ' 

d{Tix,n)/nn _ exp{-n/x)Lix) Ljx) 

dVL a;'^+2(r-l)! a;''+2(r - 1)! ' ^ ' 

The right-most parts of (|98l) and (|99l) are integrab le on (xd, 00), because of property [2] 
in Assumption 131 This implies (jFlemingl . 1977 . theorem 5.9) that fjo/^^ is a 
function of fl, and therefore so is fju] in addition, d^u/dil satisfies an expression 
analogous to (j96p with the integration interval replaced by (xc, 00). 

The preceding results assure that dfj/dfl = X^fLo d^i/dl^ is continuous and can be 
expressed as in (|T2l) . The equality (fTB)) readily follows from (fTTl) and (fT2l). □ 

Lemma 3. For any a,c E M+, 6 G M, 

Proof. Applying the change of variable x — VL/v, the integral in ()100p can be expressed 
as 

r f^^exp(-17/.),_ r^Va .,^..._o.-c/ 



■da; = ^ 17''~"iy"-iexp(-;/)di^ = 17''-" (^c,^^ , (101) 
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from which (|100l) follows. 
Lemma 4. For s G M, 



lim 



lim 



l{s,u) ^ 1 

S 



1 exp(— u) 



1. 



□ 

(102) 
(103) 



Proof. The se equalities respectively follow from lAbramowitz fc Stegun (|l970l equation 
6.5.29) and lAbramowitz fc Stegunl (|l970l equation 6.5.32). □ 

Lemma 5. The upper incomplete gamma function defined in ([3]) satisfies for s, w G N 

/ s-l 



^(5_l)(-fe-i);.^-exp(-i.), s>l, 

s-l 

(s- l)('*-'=-i)zyfcexp(-j.) + W^(zy), s<0, 



(104) 



^ k—s — w 
,s — w~l 



where W{iy) is 0{v'' ^ exp(— t^)) as v oo. 

Proof. The expression for s > 1 is equivalent to Abramowitz fc Stegun ( 1970l equation 
6.5.13). 



F or s < 0, the stated result follows from recursively using the identity (jAbramowitz fc Stegunl . 
1970l equation 6.5.21) 



r(s, i^) = (s - l)r(s + i^"'^ exp(-z^) 

w times and taking into account the equality ()103p from Lemma E) 



Lemma 6. For i E N. u E Z, 



E 

J=0 



0, i = l,...,i-l, 

(-1)*-H!, i = t. 



Proof. The equality 



(105) 

□ 

(106) 
(107) 



is easily shown to hold for A; G N by applying the binomial theorem to {x — 1)*, 
differentiating k times and particularizing for x — 1. The term j{u—j)^^~^'> in (|106p can 
be expressed as X]fe=i '^kj'^'^^ for appropriate values of the coefficients a^; furthermore, 
it is easily seen that equals (—1)*^^. Thus 



j=0 ^"'^ k=l ]=0 ^-^^ 



(108) 



If i < i — 1 , the inner sum in p08p equals for all k within the range specified in the 
outer sum, because of (|107p . li i — t, all values of the index k give a null inner sum 
except k = t, which gives attl = (-1)*"^!. This establishes piM)) . □ 
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Proof of Proposition\^ Assume that (fT7)l holds. Let e = — i3s/(4r), which is positive 
for the allowed values of B and s. From (flTl) . there exists (5 such that \L{x)'-A—Bx^\ < 
ex* for all x S (0, S). This implies that given ^ e (0, (5), and for (, < x < S, 

L(0 - ^(a;) > B(r - 2:") - 2ex'' = S^" - + 2e)a;''. (109) 

Therefore 

m~Lixi^^^ f^m-Ljx)^^^ rm-m^^ 



B B + 2e B + 2e L{C} - L(x) , 

' ' ax 



B _ B B + 2e _ B + 2e f°° - L{x) 

(110) 

Denoting by C the sum of the terms in the right-hand side of (|110p which do not 
depend on ^, i.e. the second, third and fifth, and substituting the value of e, 

^^^^l^ax>- ^ \ +a. (Ill) 

x'^+i 2r{r - s) 

Taking into account that —Bs and r — s are positive, and that C is independent of ^, 
from (|llip it is seen that there exists ^ £ (0, (5) such that (|14p holds. □ 

Lemma 7. Under the hypotheses of Theorem\^ there exists fio swc/i that d^/df2 < 
for all r2 < f^o- 

Proof. Let ^ be as in property [T] in Assumption 21 Since L is non-increasing for all x 
smaller than ^, the function £ defined as 

^ ^ [0 fora;>C 

is non-negative and non- increasing. From ([T0|) and (fTT|) . can be expressed as Co -I- 
Ci + C2 with 

POO 

Co= / c^{i^)md^. (113) 

poo 

Ci = / (/.(z^)^(r!/i.)dj., (114) 

i/'(2:,f7)L(a;)da;. (115) 

Each of these terms can be interpreted as the risk associated with a certain loss function 
for which Proposition [1] applies. 

Since £ is non-negative and non-increasing, for fixed the integrand in (|114p is a 
non-negative, non-increasing function of fl. This implies that Ci is a non-increasing 
function of fi, and thus d^i/dO < 0. 
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Regarding the term Co, 

dCo ^ n^-^eM-^/OHO 
an e(r-iy. 

which impUes that 



(116) 



As for C2, from (|115p it follows that 

dC2 r»'-i p exp(-fVx)L(x) , f°° exp{-n/x)L{x) 

dn=V^.J, ^"^(^y« ^'''^ 

Interpreting the integrals in (|118p as Lebesgue integrals, and n oting that expf — fl/x) < 
1 for D,,x € R+, Lebesgue's dominated convergence theorem ( Apostoll . Il974 theorem 
10.27) assures that 

■in..r °""-™ d.^r^d.. (11,) 



and similarly for the second integral. This implies that the first term in the right-hand 
side of (jllSp dominates the second for il asymptotically small, i.e. 

lim ^ = [ ^, } dx. (120) 

0^0 f^'^-i (r - 1)1 a;''+i ^ ^ 

From (fTTT]) and ([T20l) . 

iim-^(^"+^-)/'^"^- ,+^rMgd. 



(r-l) 



(121) 



Combining (|12ip with the inequality ([T^ from Assumption HI the limit on the right- 
hand side of (|12ip is seen to be negative. This implies that there exists fig such that 
d(Co + C2)/dr2 < for < Hq. Taking into account that d(i/dft < 0, it follows that 
dfj/dn < for < 1^0. □ 

Lemma 8. Under the hypotheses of Theorem\^ there exists fip such that dfy/dfJ > 

for all n>n'Q. 



Proof. If condition (a) of property [2] in Assumption |4] holds, let H be chosen such 
that < < L(^'+) - L(^'-). By definition of L(^'-), there exists h such that 
L{x) G {L{i'-) - H,L{i'-) + H) for all x e (C' - /i,^')- ^ condition [(b)] holds, it 
stems that there exists h such that (— l)*~^d*L/dx* is positive and continuous for 
a; G (^' — h,£,'). Thus, let h be selected as has been indicated. 

Prom property [T] in Assumption [Sj there exist K £ M., Mi, and xl < ^' — h such 
that 

L{x) < Mi^x^ for X < XL. (122) 
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The asymptotic risk rj can be expressed from pO|) and pTjl as Co + Ci + C2 + C3 + C4 
with 



ijj{x, fl)L{x) dx, 



^{x, fl)L{x) dx, 
ipix, fl)L{x) dx, 

L 

00 

tP{x,n)L(^'+)dx, 



H(L(r!H-L(e'+))di.. 



(123) 
(124) 
(125) 
(126) 

(127) 



Each of these terms corresponds to the risk associated with a certain loss function 
which satisfies Proposition [TJ 

By property[2]of Assumption [H L{x) — L{£^'+) is non-negative and non-decreasing 
for X > . An argument analogous to that used for Ci in Lemma [7] shows that the 
term ^4 given by p27p is non-decreasing with fi, and thus 



da 

dn 



> 0. 



According to Lemma |31 dC,ydn is given by 
dO 

Computing 

dC( /""^L rr2'-iexp(-r2/a;) 



dn 



x^+^r-iy. 



L{x) dx 



f (r- 1)! 



L{x) dx 



and using (jl22l) it stems that 



dCl 



dVL 



< 



r~l /-XL 



(r-1)! 



exp(— fl/x) 



dx + 



x'-+2(r-l)! 
Mi^fl'' exp{-n/x) 



{r - 1)! Jo 



X 



r+2-K 



dx. 



(128) 



(129) 



(130) 



(131) 



The integrals in can be bounded as follows. Let A = (xl + ^ ^)/(2(C' ^ ^))- 

It is seen that A and 1 — A are lower than 1. Let the function vi : U {0} i-^- M U {0} 
be defined as vi{x) = exp(— Afi/x) for x > and wi(0) = 0. Since exp(— A^2/x) — )■ 
as X — )■ 0, vi is continuous on [0, xl]. In addition, the function W2 : R U {0} 1— M U {0} 
such that 

cxp(-(l - A)rj/x) 



^2(2;) = 



(132) 



for X > and ^2(0) = is no n- negative and integrable on [0,xl]. Thus, the mean 
value theorem (|FleminsJ . Il977l p. 190) can be applied to the first integral in p3ip to 
yield: 



/" 

Jo 



exp(— Jl/x) 



dx 



vi{x)v2{x) dx ^ vi(x^) / W2(x)dx 



(133) 
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for some Xm G [0, xi]. Actually, Xm cannot be 0, because that would give in the right- 
hand side of (|133p . whereas the left-hand side is greater than 0. Thus Xm G (OjXl]- 
Similar arguments can be applied to the last integral in ()133p to obtain 



exp(-(i - x)n/x) ^ ^^!^p(-(i - ^)^^/^m) 



rr+l-K 



r+l-K 



(134) 



with Xjjj € (0,a;L]- Maximizing the right-hand side of (|134p with respect to x'^^ G 
gives 



exp(-(l - X)n/x) 



r-r+l-K 



dx < Xh 



1 -K 



1- A 



r+l-K 



exp(-(r -hi - K)) 



Combining (fT33l) and ([T35|) . 



exp(-17/a;) xlI?- + 1 - exp(-(r + 1 - X + Af7/a;,„)) 

- ((1 - A)17)'-+i-A' ■ 

The second integral in (|13ip is bounded analogously: 

exp(-17/a;) ^ a;L(r + 2 - exp(-(r + 2 - X + Af7/a;;,'J) 



„r+2-K 



((1 - A)f7)'-+2-ff 



(135) 



(136) 



(137) 



with x'^ e (0,a;L]. From ([W]) . ([^51) and ([T77| . 



dl7 



< 



MLa;Lr(r + 1 - KY+'^ 



exp(-(r + + \n/xn,)) 



(l-A)^+i-ff(r-l)! 
AfLa;L(r + 2 - X)''+2-^J7^-2 exp(-(r + 2 - X + Af7/0) 



(138) 



(l-A)'-+2-^f(r-l)! 
It is easily seen that Xni/A,a;J^/A < xi, < S^' — h. It thus follows from (|138p that 



< Qn^~^exp{-n/{^' -h)) 



(139) 



where Q is independent of 17. 

For dC2/dn, by Assumption [TJ let Af be an upper bound of L in the interval 
(^^L, C ~ argument based on the mean value theorem can also be applied here; 

in fact, it is slightly simpler than in the preceding paragraph because in this case the 
lower integration limit is greater than 0: 



dC2 



< 



Mrn 



r~l ri'-h 



(r-1)! 



exp(— ri/x) 



da; 



Mff f^'-^ exp{-n/x) 



rr+2 



dx 



Mn^'-^iC' -h-xi,) freiip{-n/x: 



(r-1)! 



with xZ,x'^' e [xlA' - h]. Therefore 



dC2 



dn 



< 



Mj^'-h-xi^) 



nejip{-n/x'^ 

////r+2 



(140) 



r+—] n"-^ exp(^n/{C - h)). 



(141) 



To compute the derivative of Ca i it is necessary to distinguish cases (a) and |(b)| of 
property [2] in Assumption 21 In case (a) since L{x) € (L(^'— ) — H,L{^'—) + H) for 
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all X e — h,^'), the mean value theorem assures that there is some 6 G [L{£,'—) 
H,L{C-) + H] such that 



<K 

dO 



L{x) dx^e 



dx 



d Q'-expi-n/x) 



{r-iy.dn 



(142) 



da;. 



Applying Lemma [3l 

dc^ en^-^ 



exp{-n/^') exp{-n/{C - h)) 



dn (r-l)! 

Using (insD, (insD, (HMD, (nm) and im . 

d77 ^ - cxp(-r!/o 



(143) 



dn 



r(r-l)! 



+ 0(f7«exp(-f^/(e'-/i))) (144) 



with q = max{r,K - 2}. Since h > and 9 < L{^'-) + H < L{^'+), from (fT44)) it 
follows that 



lim 



^exp(i]/^') d^^ ^ i(e'+)-^ 



> 0. 



n-X^y n^-^ dn J '- £,'' {r - ly. " ^' ^^"^^^ 

In case |(b)[ since d*L/da;* is continuous on (^' — h,^'), Taylor's theorem ( Apostol 



19671 volume 1, theorem 7.6) can be applied to express L{x) for a; G (^' — h,^') as 



t! 



(146) 



where 0' is the value of d*L/da:;* at some point within the interval {£,' — h, The choice 
of h assures that (— 1)'~^6'' is positive. Substituting (|146p into (|123p . differentiating 
and making use of Lemma [3] and ^ gives 



dn n{r-iy. 



{r-iyti 
n 



-i^^Jexpi-n/C)^ 



cxp(-r!/(^' - h)) 



-[-] exp{-n/a 



(^) " exp(-n/(e' -.)) + . (r J, l)-r{r- J, ^) 



(147) 



The identity ^J-^^ (*)(-l)*"^ = implies that 



23 



and thus (|147|) simplifies to 

dC^ _ L{e-)n-^ f eM-^/i') , exp(-r!/(C' - h)) 



(r-1)! 



{r-iyt\ 



J J ie - hy-i 



(r - l)!t! ^ \j 



From Lemma[5l fl^ ^r(r — j, fi/^') for j < r — 1 is given by 



(r_j_l)('-fc-i)f7fe-i 



Ik-] 



(149) 



(150) 



whereas for j > r and for any w G 



r2 



r!^-ir(r-j,-) =exp(-r!/e') 



[r-j-iy 



r — k- 



A;— r— 



(151) 



■cxp(-0/e')) 

Replacing ^' by ^' — /i in (|150l) and (|15ip it is seen that 



(^r - J, ^) = O {n^-' exp(-17/(r - h))) 



(152) 



Setting ti; = t in and substituting p ^ - (IT5^ into yields 



dc^ ^ £(e^-)r!'-icxp(-i]/go 

df^ C"^(r-1)! 



9' exp(-r2/e') 
(r- l)!t! 



min{f,r — 1} 



(r-j-i)('-fc-i)rj'=-i 



j=0 ^-^^ k=] 

r-1 



(^_j_i)(r-fc-i)f^fe-i 



/k-j 



k—r — t 



^Ik 



(153) 



(the term 0(0''~'~^ exp(— 51/^')) could be substituted by a lower-order term if i < r, 
but this is unnecessary for the proof). Since {r — j — l)(''^'''^i) = for k < j < r, 
the summation range of the first sum over k in (|153p can be extended from k = 
j,...,r— 1 to k — min{0, r — <},...,r — 1. On the other hand, the second sum over j 
is empty if t < r. Thus the second sum over k only appears ii t > r, and in this case 
min{0, r — t}~r — t. Therefore the lower limit in the latter sum can also be expressed 
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as fc = min{0,r — t}. With these changes, (|153p is rewritten as 



an f'-(r-i)! 



■ ' fe=min{0,r-t} ^ i=0 



(154) 



From Lemma |6l the inner sum in (|154p equals for fc = r — t + l,r — i + 2,...,r— 1 
and (— l)*~^t! for fc = r — If i < r, the terms with index fc = 0, l,...,r — < — 1 are 
0(17''-*-2gxp(-rj/f )). Therefore 

dC^, i(^'-)l]'-iexp(-17/e') (-l)*-iC'^*-'^6''f]'-*-iexp(-f7/e') 



dJ7 e'(^-l)! (^-1)! (155) 

+ o(r!'^-*-2gxp(-r!/e')) • 

Using p^ . (dig), (UMl), ([TiT|) and and considering that i(^'-) = 1^(C'+), 

d?7 (-l)*-iC'2*-'-0'r2'-*-i exp(-17/C') 



> 



dr2 - (^-1)! 

Since (-1)*"^6'' > 0, this imphes that 



+ 0(f^'-*-2gxp(-f7/^')) • (156) 



/exp df7\ -1 t-i£'2t-'-6'' , , 

1™ ^ / 1 S t^ > 7 >0. 157 

As a consequence of (|145p and (|157p . in either case (a) or |(b)| of property [2] in As- 
sumption HI there exists f^Q such that Af\ld£l > for > ilp. □ 

Proof of Theorem\^ From Lemmas [7| and [3 there exist i7o, such that, denoting by 
^Ifj the value of f] corresponding to a given fi, 

^In > ^Ino for n < no, (158) 
f}\,,>v\^, forn>n'o. (159) 

Proposition [T] implies that is a continuous function of fl. Therefore, this function 
restricted to the interval [Jlo, ^^o] absolute maximum (jApostoll . Il974 theorem 

4.28). Because of (jlSSp and (|159p . this is the absolute maximum of fj over R+. □ 



Lemma 9. Under the hypotheses of Theorem\3i given a G M+, C, as defined by p3p 
is a continuous function of fl £ . 

Proof. From Assumptions [T] and [2l L is continuous except possibly at a finite number 
of points, where it can only have removable discontinuities or jumps. Since removable 
discontinuities do not have any effect on the integral in (pS)) , they can be disregarded. 
Thus in the following it is assumed that L only has jump discontinuities. Let D be the 
number of discontinuity points, located at ii < X2 < ■ • ■ < xd- The function L can 
be decomposed as the sum of and Ld, where is continuous and is piecewise 
constant with jumps at xi, . . . , s/j. Accordingly, C, = Cc + Cd, where Cc and are given 
as in p3p with L replaced by and Ld respectively. 
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For any il' ^ fl, let C' denote the right-hand side of ([33]) with il replaced by ft' , 
and let Cc s-nd C'd be defined similarly. For e > arbitrary, it is necessary to find 
S > such that |C' - CI < e for |0' - fi] < (5. Cons ider an arbitra ry i5o £ (0, fi). 
Since Lc is continuous, by the Heine-Cantor theorem (lApostol 1974 . theorem 4.47) 
it is uniformly continuous on the interval [{ft — (5o)/(r(T), {il + 5o)(j/r]. This interval 
contains the values fl/v and ft'/i/ for \ ft' — ft\ < Sq, v £ [r/(T,ra]. By virtue of this, 
defining ec — e/{2r{a—l/a)), let Sc < 6o be chosen such that \Lc{ft' /i/) — Lc{ft/i/)\ < 
for all |f2' — f7| < ^c, V £ [r/cr^ra]. Taking into account Lemma [U it follows that 

|Cc-Cc|<^^ \L{n/i^)-L(n' /u)\dv <r(^a-^e,= ^- ior \n' - ft\ < 5,. {im) 

By construction, there exists an upper bound M(j on x G M+. Since 

Ld(yi/v), considered as a function of i^, has jumps at fl/xi, . . . ,ft/ xd, associated with 
each discontinuity point fl/xi there is an interval of values of v for which L(i{fl' /v) ^ 
La(SI/v). The width of this interval is - Vl\/xi < \n' - n\/xi, and \Ld{fi'/iy) - 
Ldift/iy)] < 2Md for i' within this interval. There are at most D such intervals 
contained in [r/a,ra], and for any value of ly not belonging to any of these intervals 
it holds that Ld(fi' /ly) = Ldift/v). Using Lemma [1] again, it is seen that |Cj — (d\ < 
2DMd\ft' — ft\/xi. Thus there exists 6d such that 

ICd-Cd|<| \n' - n\ < Sd. (161) 

Taking 5 — min{(5c, (5d}, it follows from (|160p and (|16ip that 

IC'-CI<lC-Cc| + |Cd-Cd|<| + |=e for\n'-n\<S, (162) 

which shows that C is a continuous function of 17. □ 
Lemma 10. Under the hypotheses of Theorem\^ and with Q defined by (j33p . 

^ — C ■ • V — C 
lim limsup = lim limsup — - — ~ 0. (163) 

"^■^ n^Q C n^oo C 

Proof. According to property [T] in Assumption l3l there exist K < r and tol, Afb, xi^ e 
R+ such that mi^x^ < L{x) < Mi^x^ for x < xl, that is, 

mi^{Vt/v)^ < L{Vt/v) < Mi,{niv)^ for v > ft/x^. (164) 

Similarly, property [5] implies that there exist K' < r; m^, e M^; and > xl such 
that 

m'^{n/iy)^' < L{n/v) < Ml{n/v)^' for v < n/x'^. (165) 

From Assumption [1] L is of bounded variation on [xl,Xl], and thus there exists M 
such that L{x) < M for x E [xl, x'-^], that is, 

L{n/v) < M for Vt/x'^ <v< VL/xi^. (166) 

The case is analyzed first. Given a G M+, it will be assumed that Vt < rxi^ja. 
Under this assumption, any v within the integration interval in p3p exceeds 
Thus, applying (I164p . 

/- t.'-^-iexp(-z.) ^ m^^^{V{r-K,Tla)-V{r-K,ra)) 



26 



The difference ^ — C can be expressed Ci + C2 + Ca + C4i where each term is 
an integral as in p3p with the integration interval respectively given as {Q^VL/x'^), 
(JI/xl, ^^/xl), (ri/xL, J'/ct) and (rcr, 00). In the first case, ()165p implies that 

(r- 1)! 

and thus 

C mL(r(r - K, r/a) - r(r - K, ra)) ' ^ ' 

Using the equality (|102p from Lemma |31 and taking into account that K,K' < r by 
Assumption [3l it is seen that the right-hand side of (|169p tends to as — > 0. Since 
Ci and C, are both positive, this implies that 

lim ^ = 0. (170) 
n-).o ( 



As for the term (2, using (|166p . 

(r-1)! " (^--l) 



< M(7(r,r!/xL)-7(r-,»/0) ^ M7(r, fj/a^L) ^ ^^^^^ 



and thus 

C mi^{r{r-K,r/a)-r{r-K,ra))' ^ ' 

Using (|102p again, and taking into account that K < \i stems that 

lim ^ = 0. (173) 

Regarding the third term, (|164p holds for all v within the integration interval, and 
thus 

, ^ MLn^(7(r - K, r/o) - 7(r - K, ^/.tl)) ^ MLn^7(r - K, r/o) 

< < ^~T)^ ■ 

Therefore 

C3 ML7(r-X,r/a) 

C TOL(r(r-K,r/cr)-r(r-X,rcr))' ^ ' 



Similarly, the fourth term satisfies 

(r-1) 



a < ^-^""^r-Kra)^ ^^^^^ 



and therefore 

Ci ^ M^rir-K,ra) ^^^^^ 

^ 77iL (r (r — if , r/cr) — r(r — if, rcr)) 

From PTO)) . P75)) . p75)) and p77)) it follows that 

hmsup— ^< -— — — r-. (178) 

n^o C mi^{T{r - K,r/a) -r(r - K,ra)) 



^Note that this decomposition, and the one to be used for f! — > 00, are different from those used 
in the proofs of Lemmas [7] and \E\ respectively, although the same notation is used for simplicity. 
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The right-hand side of (|178p is seen to converge to as cr —?> oo, and thus so does the 
left-hand side. This estabhshes the first part of the result. 

The analysis for J7 — >■ oo is similar. Given a € M"*", it is assumed that > rx'^^a. 
The difference 77 — C is expressed as Ci + C2 + C3 + C4J where each term is an integral as in 
with integration intervals respectively given as (0, r/a), (rcr, 51/2;^), (^1/2;^, ^I/xl) 
and (ri/xL, 00). Arguments analogous to those used for — establish that 

hm sup < ^''^^^J - ^ ' + - ^ ' . (179) 

The right-hand side of (|179p is seen to converge to as cr — >■ 00, and thus so does the 
left-hand side. This establishes the second part of the result. □ 

Lemma 11. Under the hypotheses of Theorem\^ considering C, and fj as functions of 
Q G K+, (^/t] — > 1 uniformly on a,s cr — > 00. 

Proof. The result is equivalent to the statement that for any e > there exists ctq such 
that jry/C— 1| < e for all £ and for all cr > ctq. Consider e > arbitrary. Let i?(cr) 
and R'{a) respectively denote lim supjj^g (?/ — C) /C and limsups^_j.o2(77 — C)/C- Since L 
is a non-negative function, from p3p it is seen that C is a non-negative, non-decreasing 
function of a for any fl. By Lemma [TUl i?(cr) and R'icr) tend to as cr 00, and thus 
there exists cri such that i?(cri), i?'(cri) < e/2. By definition of R{a), there exists fio 
such that the following inequality holds (note that the left-hand side is a function of 
cr and f2): 

< R{<7i) + ^ < e for n <no, cr = (Ti. (180) 

The non-decreasing character of C with cr implies that (|180|) also holds for cr > cti, 
that is, 

< e for O < Oo, cr > cti. (181) 
Analogously, there exists fig > ^0 such that 

rl-C 



C 



< e for > cr > cti. (182) 



According to Lemma [9l for a fixed, C is a continuous function of $7 G [flo, I^q], and 
therefore it has an absolute minimum on that interval, which will be denoted as 5*1 (cr) . 
The non-negative and non-decreasing character of C with cr implies that Si is also a 
non- negative, non-decreasing function. In addition. Si (cr) > for all cr greater than a 
certain value cr2. This can be seen as follows. By Assumption [3^ L{x) is non-zero for 
all X outside a bounded interval. If a is sufficiently large, i.e. greater than a certain 
cr2, for any Q G [r^Oji^o] the integration interval in p3p contains a subinterval where 
L is non-zero, which gives ( > 0. Thus S'i(cr) > for a > cr2. 

By arguments similar to those in the above paragraph, fj — considered as a 
function of fi, has an absolute maximum on [rio,fio]; and this maximum, denoted as 
S'2(cr), tends to as cr 00. Therefore, defining S{a) — S2{<j)/ Si{a) for a > 02, 

{fj - C)/C < S{a) for n G [r!o, %], a> a^; (183) 

and <S'(cr) as cr 00. Thus, for the considered e, there exists > (J2 such that 
S{a) < e for cr > 0-3. Combined with (|183p . this gives 

[fj - C)/C < e for r! G [l^o, n'^], a > a^. (184) 
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From (|18ip . (|182p and (|184p . choosing o-q = niaxjcri, 0-3} is sufficient to satisfy \rj/C, — 
1| < e for Q, e R+, ct > ctq. This completes the proof. □ 

Proof 0/ Theorem O The result will be proved by contradiction. Assume that there 
exists a possibly randomized estimator p with limsupp^Q 77 (p) < 77*. This implies that 
there exist 6 < 1 and a probability pq such that the estimator has 

ri{p) < 9r]* for ah p < pe. (185) 

For n = r, r + 1, . . ., let n„ denote the distribution function of p conditioned on = n. 
By Lemma [Tl] let a be selected such that 



v)L{Vl/v) dv> </e I (t>{v)L{n/iy) for all Q e R+. (186) 

r/a Jo 

In particular, this implies that 

(j){v)L{Q.lv) dv > \/er]* for aU n e IR+. (187) 

r/(7 

Given Ui,U2 with 1/2 > vi > 0, according to Lemma [2l $(p, i^) — >■ (f>{i') uniformly 
on [vi, 1^2] as p — >■ 0. By virtue of this, let pi < pg be such that 

\<l>{p,v)- < {I- \/e)(t){v) for aUp <pi, 1/ e [r/cr,rcr]. (188) 

Let u = \ra/pi~\. Taking into acco unt that lim,„^no(y^™ ^ 1/n — \ogw) = 7, where 7 
is the Euler-Mascheroni constant ( Abramowitz fc Stegun . 1970l equation 6.1.3), it is 
easy to see that 




1 ^ 

= 7 + log— (189) 



api n 

n—l 

This implies that there exist A > and p^ such that 

Lr/(ap)J ^ 

log 

p 

Let A and p'q be chosen such that (|190p holds, and let be defined by the equation 



- - log — > -A for aU p < p'^. (190) 



Since A > and < 1, it follows that p^ < pi. 

Let po = min{po,j)g}. For a given n, the measure associated wit h the distribution 
function n„ is obviously finite, and thus sigma- finite. This implies (jBillingslevl Il995 . 
theorem 18.3) that for each n the integral in ([3]), considered as a function of p, is 
measurable with respect to Lebesgue measure. In addition, since p i < pe, it i s clear 
from (|185p that the series in ^ converges for p <pi. This assures ([Billingslev . 1995 



theorem 13.4(ii)) that ri{p) restricted to p < pi is measurable. Therefore, the integral 

X=r!Map (192) 
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exists in the Lebesgue sense, and according to (|185p it satisfies 



X < r ^ = Of]* log—. (193) 



Substituting ^ into (fT92)) . 

rpi 1 oo / i-oo \ 

- E / ^^yM dn„(y) dp. 
Defining w = [r/(crpo)J, it is clear from (|194l) that 



(194) 



(195) 



Since both measures in (|195p are sigma-finite, and b oth the inn er and outer integrals 
are finite, the order of integration can be reversed (|BillingsleyL Il995i . theorem 18.3), 
which gives 

Making the change of variable v = np in the inner integral and taking into account 
that f{n)/p= ^{p,np), (I196P becomes 

For w < n < w it holds that np^ < r / a and npi > ra. Therefore 

<i>{iy/n,i^)L{ny/iy)diy]dnn{y). (198) 



1 poo / pro- 



For v € [r/a, ra] and u < n < v it holds that i//n < pi. Thus (|188p gives $(i^/n, ;/) > 
\f9(t){v). Substituting into (jTMl) . 

X>^V-/ / 0(i.)L(n2//i.)di. dn„(2/). (199) 

From (|187p . the inner integral in (|199p exceeds \/9ri* , and thus 

^ 1 

X > 6*2/3,^* ^ -. (200) 

n—u 

Since ^ J'o and ]9o ^ Po < Pi, f|190P and (|19ip give 

E - > + l°g - ^ l°g - fl - - (201) 

^ " Po Po V log(pi/po)/ Po 



n— u 



Substituting into ([200| . 
in contradiction with (|193p . This establishes the result. □ 



X>9'q*\og^, (202) 
Po 
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Proof of Proposition [21 The proof is analogous to that of Mendo &: Hernandol |2009l, 



proposition 1). 

Proof of Proposition^ For the considered estimator, 



E[(p-p)2 



(r-2) 



The equality 



■E 



E 



1 



1 



2(r-2) 



E 



N-1 



+ 1. 



TV- 1 



□ 



(203) 



(204) 



directly stems from the fact that ([TJ is unbiased. On the other hand, according to 
Mikulski fc SmithI (Il976h . for p e (0, 1) 



Var 



A^- 1 



From (|204l) and ([SOS]) . 

1 



E 



(A^- 1)2 



= E 



1 



n \ 2 



iV- 1 



Var 



N- 1 



< 



(r-l)(r-2)- 



Substituting (p04)) and ((206)) into ((203l) . the desired resuh is obtained. 



(205) 



(206) 



□ 



Lemma 12. Given r > 3, considering the loss function (jl9p iwit/i Ai = 0, > 0, if 
A*2 > (r + Y^+l)/f2 t/ie risk of the estimator ()28p satisfies ri{p) < fj for any p € (0, 1). 
Similarly, for the loss function (|19p wit/i = 0, ^1 > 0, i/ /xi > ri/(r — ^/r) the 
inequality ri{p) < fj holds for any p e (0, 1). 

Proof. The stated results follow from the arguments used in the proof of Mendo fc Hernandol 
( 20091 proposition 3). □ 



Proof of Proposition [31 The result will be proved by approximating the loss function 
as a sum of terms of the form ([T9|) with Ai,A2 > and using Lemma [121 It may be 
assumed without loss of generality that L{x) = for x e because if L{x) = C 

within that interval, defining L'{x) — L{x) — C the risk corresponding to L is expressed 
as C plus the risk resulting from the loss function L', which satisfies the hypotheses 
of the Proposition. 

Let e > 0, and suppose for the moment that L is unbounded on the interval (0, v). 
This implies that for any z S N, the set V^^i — {x € (0,-u) | L{x) > ie) is non-empty. 
In fact, since L is non-increasing on (0,u), V^^i is an interval. Let x^^i be defined as 
the supremum of V^^i, and let 



e li X < Xe.i, 
otherwise. 



(207) 



If L is bounded on (0, v), the sets V^^i are empty for i greater than a certain value. In 
this case, the corresponding ^^.i functions are defined as the null function. In a similar 
manner, for L unbounded on (u',00), let V/j = {x € (u',00) | L{x) > ie}, which is 
again non-empty interval; let x[ ^ be its infimum, and 



e li X > x'^ ^, 
otherwise. 



(208) 
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If L is bounded on (u',oo), for i greater than a certain value the sets V^^ are empty, 
and the corresponding l'^^ are defined as null. Let L^_i{x) = £e,i{x) +£[^{x) and 
L^{x) — X^i^i By construction, for all x e M+, 

Q < L{x) - L,{x) < e. (209) 

Each function L^^i satisfies Assumptions [TH3l and therefore a risk can be defined 
considering L^^i as the loss function. This risk will be denoted as ri^^i{p). The function 
Le also satisfies Assumptions HHS] Let 77^ {p) denote its corresponding risk. 



00 00 



V.{P) = E f{n)LMn)/p) I{n)LMn)/p) (210) 

n—r n—r i—1 

For each n, the inner series in ()210p converges absolutely; namely, to f{n)L^{g{n)/p). 
In addition, from (|209p it is seen that L^{g{n)/p) < L{g{n)/p), and this implies that 
the outer series in (I210II is also ab solutely convergent. This allows interchanging the 
sums over n and i ( Apostol . 1974 theorem 8.43), which gives 



r7,(p) =Er7,,,(p). (211) 

Theorem [1] assures that rie,i{p) has an asymptotic value ?7e^i, given by 

(j){iy)L,4n/iy)diy, (212) 
Similarly, r}^{p) has an asymptotic value 

fi,= / <j>{i^)y2L,4n/iy)diy. (213) 

Since j is a n onnegative function for all i, the monotone convergence theorem 
( Athreva &: Lahiri, .2006k theorem 2.3.4) implies that the sum and integral signs in 
(|213p commute, and thus 

00 

=$]^e,». (214) 

i=l 

From Lemma [T^ Ve,i{p) < Ve.i- Combined with (I21ip and (|214l) . this gives 

%{p)<%- (215) 
On the other hand, from (|209p it stems that 

< V{P) - V.{P) < (216) 

which in turn implies 

< r? - rye < e. (217) 

From (|2T5 l) - ([2T7|) . 

Vip) ^ Veip) + <^ < rjc + £ < rj + (218) 
Since (|218p holds for e arbitrary, the desired inequality 'i]{p) < fj follows. □ 
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